Skip to content

Draft: <fix>[zbs]: ZSTAC-86782 backport MDS failover#4578

Open
ZStack-Robot wants to merge 2 commits into
5.4.12from
sync/haidong.pang/fix/ZSTAC-86782
Open

Draft: <fix>[zbs]: ZSTAC-86782 backport MDS failover#4578
ZStack-Robot wants to merge 2 commits into
5.4.12from
sync/haidong.pang/fix/ZSTAC-86782

Conversation

@ZStack-Robot

@ZStack-Robot ZStack-Robot commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Backport the complete ZSTAC-80595 fix for CBD/ZBS volume-clients anti-split-brain checks. When the selected MDS is unreachable, the request now uses the existing next-MDS path instead of ending before failover can run.

Source provenance

  • zstack !9153 / 3b5bda3b76aef968a911d18e35b3b30bd0cab803: use the synchronous MDS caller with a 30-second timeout and enable tryNext.
  • zstack !10320 / 1f65e4e5a8a547176cab7c8ed029e3288765d9f6: convert synchronous REST failures into failed agent responses and add the regression case.

Changes

  • Keep /zbs/primarystorage/volume/clients timeout at 30 seconds and enable next-MDS handling.
  • Let synchronous network failures reach the existing failed-response branch.
  • Backport the existing ZbsPrimaryStorageCase regression scenario; the only cherry-pick conflict was target-branch import context.

Testing

  • git diff --check upstream/5.4.12..HEAD
  • Source and backport production patch-id match for the final sync-failure change.
  • Remote x86 plugin/zbs package
  • Remote x86 ZbsPrimaryStorageCase

Resolves: ZSTAC-86782

sync from gitlab !10547

@coderabbitai

coderabbitai Bot commented Jul 16, 2026 •
edited
Loading

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 13 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 21665d99-83b3-4e5f-bdff-497c22ef6ab5

📥 Commits

Reviewing files that changed from the base of the PR and between dfbed98 and 839f494.

📒 Files selected for processing (3)
  • plugin/zbs/src/main/java/org/zstack/storage/zbs/ZbsMdsBase.java
  • plugin/zbs/src/main/java/org/zstack/storage/zbs/ZbsStorageController.java
  • test/src/test/groovy/org/zstack/test/integration/storage/primary/addon/zbs/ZbsPrimaryStorageCase.groovy

Warning

.coderabbit.yaml has a parsing error

The CodeRabbit configuration file in this repository has a parsing error and default settings were used instead. Please fix the error(s) in the configuration file. You can initialize chat with CodeRabbit to get help with the configuration file.

💥 Parsing errors (1) 
Could not fetch remote config from http://open.zstack.ai:20001/code-reviews/zstack-cloud.yaml: TimeoutError: The operation was aborted due to timeout
⚙️ Configuration instructions
  • Please see the configuration documentation for more information.
  • You can also validate your configuration using the online YAML validator.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch sync/haidong.pang/fix/ZSTAC-86782

Comment @coderabbitai help to get the list of available commands.

AlanJager and others added 2 commits July 17, 2026 17:27
@AlanJager
<fix>[zbs]: enable tryNext and 30s timeout for getActiveClients MDS call
782c2d2 
When anti-split-brain check selects a disconnected MDS node, the HTTP
call now times out after 30s instead of 5+ minutes, and automatically
retries the next available MDS via tryNext mechanism.

Resolves: ZSTAC-80595

Change-Id: I1be80f1b70cad1606eb38d1f0078c8f2781e6941
<fix>[zbs]: try next mds on sync failure
839f494 
Catch sync REST failures in the ZBS MDS HTTP caller so getActiveClients can try the next connected MDS when the selected MDS is unreachable. Add a regression test for the volume clients path where the first MDS throws an operation failure and the second returns active clients.

Resolves: ZSTAC-80595

Change-Id: Id891693b033ac3d4588243d2dd25750f63110db6
@MatheMatrix
MatheMatrix force-pushed the sync/haidong.pang/fix/ZSTAC-86782 branch from bff8a74 to 839f494 Compare July 17, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ZStack-Robot @AlanJager